“Virtual reality” is a term that has been used for various types of content that simulates immersion in a three-dimensional (3D) world, including, for example, various video game content, and animated film content. In some types of virtual reality, a user can navigate through a simulation of a 3D environment generated based on the computer model, by controlling the position and orientation of a virtual camera that defines a viewpoint for a 2D scene that is displayed on a two-dimensional display screen. A variation of these technologies is sometimes called “augmented reality.” In an augmented reality setup, the display technology shows a combination of the user's surroundings that is “augmented” by one or more digital objects or overlays. Augmented reality content may be as simple as textual “heads up” information about objects or people visible around the user, or as complex as transforming the entire appearance of the user's surroundings into a fantasy environment that corresponds to the user's real surroundings.
Virtual reality (VR) and augmented reality (AR) have been applied to various types of immersive video stereoscopic presentation techniques including, for example, stereoscopic virtual reality headsets. Headsets and other presentation methods immerse the user in a 3D scene. Lenses in the headset enable the user to focus on a lightweight split display screen mounted in the headset only inches from the user's eyes. Different sides of the split display show right and left stereoscopic views of video content, while the user's peripheral view is blocked. In another type of headset, two separate displays are used to show different images to the user's left eye and right eye respectively. In another type of headset, the field of view of the display encompasses the full field of view of eye including the peripheral view. In another type of headset, an image is projected on the user's retina using controllable small lasers, mirrors or lenses. Either way, the headset enables the user to experience the displayed virtual reality content more as if the viewer were immersed in a real scene. In the case of augmented reality (AR) content, the viewer may experience the augmented content as if it were a part of, or placed in, an augmented real scene.
These immersive effects may be provided or enhanced by motion sensors in the headset that detect motion of the user's head, and adjust the video display(s) accordingly. By turning his head to the side, the user can see the virtual reality scene off to the side; by turning his head up or down, the user can look up or down in the virtual reality scene. The headset may also include tracking sensors that detect position of the user's head and/or body, and adjust the video display(s) accordingly. By leaning or turning, the user can see the virtual reality scene from a different point of view. This responsiveness to head movement, head position and body position greatly enhances the immersive effect achievable by the headset. The user may be provided the impression of being placed inside or “immersed” in the virtual reality scene. As used herein, “immersive” generally encompasses both VR and AR.
Immersive headsets and other wearable immersive output devices are especially useful for game play of various types, which involve user exploration of a modelled environment generated by a rendering engine as the user controls one or more virtual camera(s) using head movement, the position or orientation of the user's body, head, eye, hands, fingers, feet, or other body parts, and/or other inputs. To provide an immersive experience, the user needs to perceive a freedom of movement that is in some way analogous to human visual perception when interacting with reality. Content produced for VR can provide this experience using techniques for real-time rendering that have been developed for various types of video games. The content is may be designed as a three-dimensional computer model with defined boundaries and rules for rendering as video output. This content can be enhanced by stereoscopic techniques to provide stereoscopic output, sometime referred to as “3D,” and associated with a VR application that manages the rendering process in response to movement of the VR headset, to produce a resulting VR experience. The user experience is very much like being placed inside a rendered video game.
In other types of VR and AR, the simulated 3D environment may be used primarily to tell a story, more like traditional theater or cinema. In this type of VR or AR, the added visual effects may enhance the depth and richness of the story's narrative elements or special effects, without giving the user full control (or any control) over the narrative itself. However, the technology for experiencing anything similar to cinematic content delivered using VR or AR equipment or methods is in a very early stage of development. Actual implementations of technology are quite limited, and users have thus far been largely or completely untouched by VR or AR in their experience of narrative content.
It would be desirable, therefore, to develop new methods and other new technologies for mastering cinematic content for VR and AR use, that overcome these and other limitations of the prior art and enhance the appeal and enjoyment of narrative content for new immersive technologies such as VR and AR.